Course 003: Basic Econometrics

Course 003: Basic Econometrics, 2016

Course 003: Basic Econometrics

Rohini Somanathan- Part 1

Delhi School of Economics, 2016

Page 0 Rohini Somanathan

Course 003: Basic Econometrics, 2016'

&

$

%

Outline of the Part 1

Main text: Morris H. DeGroot and Mark J. Schervish, Probability and Statistics, fourth edition. I

also draw on Introduction to Probability by Joseph Blitzstein and Jessica Hwang.

1. Probability Theory: Chapters 1-6

• Probability basics: The definition of probability, combinatorial methods, independent

events, conditional probability.

• Random variables: Distribution functions, marginal and conditional distributions,

distributions of functions of random variables, moments of a random variable,

properties of expectations.

• Some special distributions,laws of large numbers, central limit theorems

2. Statistical Inference: Chapters 7-10

• Estimation: definition of an estimator, maximum likelihood estimation, sufficient

statistics, sampling distributions of estimators.

• Hypotheses Testing: simple and composite hypotheses, tests for differences in means,

test size and power, uniformly most powerful tests.

• Nonparametric Methods



&

$

%

Administrative Information

• Internal Assessment: 25% for Part 1

1. Two midterm exams: 8% each, September 26, October 24.

2. Lab exam: 5%, November 5.

3. Problem Sets : 4% must hand in during tutorials (not accepted later).

• Tutorials: Each group will have one tutorial per week and a lab session every fortnight.

• Punctuality is critical - coming in late disturbs the rest of the class



&

$

%

Why is this course useful?

• We (as economists, citizens, consumers, exam-takers) are often faced with uncertainty. This

may be caused by:

– randomness in the world -rain, sickness, future output

– incomplete information about a realized state of the world -Is a politician’s promise

sincere? Is a firm telling us the truth about a product? Has our opponent been dealt a

better hand of cards? Is a prisoner guilty or innocent?

• By putting structure on this uncertainty, we can arrive at

– decision rules: firms choose techniques, doctors choose drug regimes, electors choose

politicians- these rules have to tell us how best to incorporate new information.

– estimates of empirical relationships such as those between wages and education, drugs

and health, money supply and GDP

– tests of hypothesis: does smoking increase the chance of lung cancer?

• Probability theory puts structure on uncertain events. The field of statistics helps us design

the collection of data samples and make use them to make inferences about the population.



&

$

%

A motivating example: gender ratios

• Does the gender ratio in a population reflects discrimination, either before of after birth?

• We visit a village and count the number of children below the age of 1.

• With no discrmination, the number of girls X ∼ Bin(n, .5).

• Suppose we decide that there is discrimination when the sample proportion, p̂ is below

some threshold.

• The following table shows the probability of false rejection of the null hypothesis p = .5 for

different sample sizes and rejection rules:

Table 1: P(Xn ≤ x) when p = .5

p̂ = .1 p̂ = .2 p̂ = .3 p̂ = .4 p̂ = .5

sample size

20 .0002 .0059 .0577 .2517 .5881

40 0 .0001 .0083 .1341 .5627

60 0 0 .0013 .0775 .5513

80 0 0 .0002 .0465 .5445

100 0 0 0 .0284 .5398



&

$

%

Gender ratios and sample size

0.0

5.1

.15

.2fre

quen

cy

0.0

2.0

4.0

6.0

8fre

quen

cy

Binomial probabilities for n=20 and 100, p =.5

As the sample size increases, there is an increase in the threshold below which the null

hypothesis of no discrimination is falsely rejected no more than 5% of the time:

binomial(20,5,.5)=.021, binomial(100,42,.5)=.067 and binomial(100,41,.5)=.044



&

$

%

Some terminology

• An experiment is any process whose outcome is not known in advance with certainty. These

outcomes may be random or non-random, but we should be able to specify all of them and

attach probabilities to them.

Experiment Event

10 coin tosses 4 heads

select 10 LS MPs one is female

go to your bus-stop at 8 bus arrives within 5 min.

• A sample space is the collection of all possible outcomes of an experiment.

• An event is a certain subset of possible outcomes in the space S.

• The complement of an event A is the event that contains all outcomes in the sample space

that do not belong to A. We denote this event by Ac

• The subsets A1,A2,A3 . . . . . . of sample space S are called mutually disjoint sets if no two of

these sets have an element in common. The corresponding events A1,A2,A3 . . . . . . are said to

be mutually exclusive events.

• If A1,A2,A3 . . . . . . are mutually exclusive events such that S =A1 ∪A2 ∪A3 . . . . . . , these are

called exhaustive events.



&

$

%

Example: 3 tosses of a coin

• The experiment has 23 possible outcomes and we can define the sample space S = {s1, . . . ,s8}

where

s1 =HHH, s2 =HHT s3 =HTH, s4 =HTT , s5 = THH, s6 = THT , s7 = TTH, s8 = TTT

• Any subset of this sample space is an event.

• If we have a fair coin, each of the listed events are equally likely and we attach probability 18

to each of them.

• Let us define the event A as atleast one head. Then A = {s1, . . . ,s7}, Ac = {s8}. A and Ac are

exhaustive events.

• The events exactly one head and exactly two heads are mutually exclusive events.



&

$

%

The concept of probability

A probability is a number attached to some event which expresses the likelihood of the event

occurring.

• How are probabilities assigned to events?

– By thinking about all possible outcomes. If there are n of these, all equally likely, we

can attach numbers 1n to each of them. If an event contains k of these outcomes, we

attach a probability kn to the event. This is the classical interpretation of probability.

– Alternatively, imagine the event as a possible outcome of an experiment. Its probability

is the fraction of times it occurs when the experiment is repeated a large number of

times. This is the frequency interpretation of probability

– In many cases events cannot be thought of in terms of repeated experiments or equally

likely outcomes. We could base likelihoods in this case on what we believe about the

world subjective probabilities. The subjective probability of an event A is a real number

in the interval [0, 1] which reflects a subjective belief in the validity or occurence of event

A. Different people might attach different probabilities to the same events. Examples?

• We formalize this subjective interpretation by imposing certain consistency conditions on

combinations of events.



&

$

%

Finite and infinite sample spaces

• Our goal is to assign probabilities to events in S.

• If S is finite, we can consider all possible subsets in S. If S has n elements, there are 2n

possible subsets. The set of all these subsets is called the power set of S.

• When S is infinite (such as the time we have to wait for a letter to arrive after an

interview), it is not obvious how we should do this.

• Carefully defining which subsets can be assigned probabilities leads to the concept of a

measurable set and σ-algebra.



&

$

%

σ-algebras and Borel sets

Definition: A σ-algebra on S is a collection F of subsets of S such that

1. ∅ ∈ F

2. If A ∈ F, then Ac ∈ F

3. If A1,A2, · · · ∈ F, then⋃∞j=1Aj ∈ F

In words: F contains ∅ and is closed under complements and countable unions.

Intuition: If it makes sense to talk about the probability of an event happening, it makes sense to

talk about it not happening, and if a set of events can occur, then at least one them can occur!

Definition: A Borel σ-algebra B on R is defined to be the σ-algebra generated by all open intervals (a,b)

with a,b ∈ R. A Borel set is a set in the Borel σ-algebra.

Analogously, we define the Borel σ-algebra on Rn to be the σ-algebra in Rn generated by open

boxes or rectangles in Rn



&

$

%

Probability spaces and axioms

Definition: A probability space is a triple (S,F,P), with S a sample space, F a σ-algebra on S and P, a

probability measure defined on F

Definition: A probability measure is a function on F, taking values between 0 and 1 such that:

1. P(∅) = 0, P(S) = 1

2. P(⋃)∞j=1Aj) =

∞∑j=1

P(Aj) if the Aj are disjoint events. (countable additivity)

We will usually use P(Aj) rather than P(Aj)



&

$

%

Probability measures... some useful results

We can use our axioms to derive some useful results that are easily proved using the axioms:

1. For each A⊂ S, P(A) = 1−P(Ac)

2. P(∅) = 0

3. For A1,A2 ∈ S such that A1 ⊂A2, P(A1)≤ P(A2)

A2 =A1 ∪ (Ac1 ∩A2). Since these are disjoint, we use our second axiom to write P(A2) = P(A1)+P(Ac

1 ∩A2).

Since both these probabilities are non-negative P(A2)≥ P(A1).

4. For each A⊂ S, 0≤ P(A)≤ 1

5. If A1 and A2 are subsets of S then P(A1 ∪A2) = P(A1)+P(A2)−P(A1 ∩A2)

As before, the trick is to write A1 ∪A2 as a union of disjoint sets and then add the probabilities associated

with them.

A1 ∪A2 = (A1 ∩Ac2 )∪ (A1 ∩A2)∪ (A2 ∩Ac

1 )

but A1 = (A1 ∩Ac2 )∪ (A1 ∩A2) and A2 = (A2 ∩Ac

1 )∪ (A1 ∩A2), so

P(A1)+P(A2) = P(A1 ∩Ac2 )+P(A1 ∩A2)+P(A2 ∩Ac

1 )+P(A1 ∩A2)

Subtracting P(A1 ∩A2) gives us the expression we want.

6. For a finite number of events, we have:

P(

n⋃i=1

Ai) =

n∑i=1

P(Ai)−∑i<j

P(AiAj)+∑i<j<k

Pr(AiAjAk)− ...(−1)n+1P(A1A2 . . .An)



&

$

%

Examples

1. Consider two events A and B such that Pr(A) = 13 and Pr(B) = 1

2 . Determine the value of

P(BAc) in each of the following cases: (a) A and B are disjoint (b) A⊂ B (c) Pr(AB) = 18

2. Consider two events A and B, where P(A) = .4 and P(B) = .7. Determine the minimum and

maximum values of Pr(AB) and the conditions under which they are obtained?

3. A point (x,y) is to be selected from the square containing all points (x,y), such that

0≤ x≤ 1 and 0≤ y≤ 1. Suppose that the probability that the point will belong to any

specified subset of S is equal to the area of that subset. Find the following probabilities:

(a) (x− 12)

2 +(y− 12)

2 ≥ 14

(b) 12 < x+y < 3

2

(c) y < 1− x2

(d) x = y

answers: (1) 1/2, 1/6, 3/8 (2) .1, .4 (3) 1-π/4, 3/4, 2/3, 0



&

$

%

Counting methods: the multiplication rule

• A sample space containing n outcomes is called a simple sample space if the probability

assigned to each of the outcomes s1 . . . ,sn is 1n . Probability measures are easy to define in

such spaces. If the event A contains exactly m outcomes, then P(A) = mn

• Counting the number of elements in an event and in the sample space can be laborious and

sometimes complicated - we’ll look at ways to make our job easier

• The multiplication rule: Sometimes we can think of an experiment being performed in

stages, where the first stage has m possible outcomes and the second n outcomes. The total

number of possible outcomes is then mn (e.g. a sandwich can have brown or white bread

and then a tomato or cheese filling)

W

B

t

c



&

$

%

Samples, arrangements and combinations

• Suppose we are making k choices from n objects with replacement. There are nk possible

outcomes

• How many arrangements of k objects from a total of n distinct objects can be had if we are

sampling without replacement? The first object can be chosen in n different ways, leaving

(n− 1) objects so the second one can be picked in (n− 1) different ways....

• The total number of permutations of n objects taken k at a time is then

Pn,k = n(n− 1) . . . (n− k+ 1)

and Pn,n = n!. Pn,k can alternatively be written as:

Pn,k = n(n− 1).. . . . (n− k+ 1) = n(n− 1).. . . . (n− k+ 1)(n− k)!

(n− k)!=

n!

(n− k)!

• How many different subsets of k elements can be chosen from a set of n distinct elements?

Think of permutations as arising by first picking k elements and then organizing them in k!

ways. Then the number of permutations is given by Pn,k = k!Cn,k, or

Cn,k =Pn,k

k!=

n!

k!(n− k)!

This is called the binomial coefficient.



&

$

%

An application: the birthday problem

• You go to watch a cricket match with a friend.

• He would like to bet Rs. 100 that among the group of 23 people on the field (2 teams plus a

referee) at least two people share a birthday

• Should you take the bet?

• What is the probability that out of a group of k, at least two share a birthday?

– the total number of possible birthdays is 365k

– the number of different ways in which each of them has different birthdays is 365!(365−k)!

(because the second person has only 364 days to choose from, etc.). The required

probability is therefore p = 1− 365!(365−k)!365k

• It turns out that for k = 23 this number is .507, so you have a small expected monetary gain

from the bet - if you don’t like risk you probably should not take it



&

$

%

The multinomial coefficient

• Suppose we have k types of elements (jobs, modes of transport, methods of water

filtration..) and want to find the number of ways that these can be chosen by n people such

that for j = 1, 2, . . . ,k jth group contains exactly nj elements.

• The n1 elements for the first group can be chosen in(nn1

)ways, the second group is chosen

out of (n−n1) elements and this can be done in(n−n1n2

)ways...The total number of ways of

dividing the n elements into k groups is therefore(nn1

)(n−n1n2

)(n−n1−n2

n3

). . .(nk−1+nknk−1

)• This can be simplified to n!

n1!n2!...nk! This expression is known as the multinomial coefficient.

• Examples:

– An student organization of 1000 people is picking 4 office-bearers and 8 members for its

managing council. The total number of ways of picking this groups is given by 1000!4!8!988!

– 105 students have to be organized into 4 tutorial groups, 3 with 25 students each and

one with the remaining 30 students. How many ways can students be assigned to

groups?



&

$

%

Independent Events

Definition: Let A and B be two events in a sample space S. Then A and B are independent

iff P(A∩B) = P(A)P(B). If A and B are not independent, A and B are said to be dependent.

• Events may be independent because they are physically unrelated -tossing a coin and rolling

a die, two different people falling sick with some non-infectious disease, etc.

• This need not be the case however, it may just be that one event provides no relevant

information on the likelihood of occurrence of the other.

• Example:

– The event A is getting an even number on a roll of a die .

– The event B is getting one of the first four numbers.

– The intersection of these two events is the event of rolling the number 2 or 4, which we

know has probability 13 .

– Are A and B independent? Yes because P(A)P(B) = 1223 = 1

3 But why?

• If A and B are independent, then A and Bc are also independent as are Ac and Bc. Show

this.



&

$

%

Independent Events..examples and special cases

1. A company has 100 employees, 40 men and 60 women. There are 6 male executives. How

many female executives should there be for gender and rank to be independent?

solution: If gender and rank are independent, then P(M∩E) = P(M)P(E). We can solve

for P(E) asP(M∩E)P(M) = .06

.4 = .15. So there must be 9 female executives.

2. The experiment involves flipping two coins. A is the event that the coins match and B is

the event that the first coins is heads. Are these events independent?

solution: In this case P(B) = P(A) = 12 ( {H,H} or {T,T}) and P(A∩B) = 1

4 , so yes, the

events are independent.

3. Suppose A and B are disjoint sets in S. Does it tell us anything about the independence of

events A and B?

4. Remember that disjointness is a property of sets whereas independence is a property of the

associated probability measure and the dependence of events will depend on the probability

measure that is being used.



&

$

%

Independence of many events

Definition: For n events, A1,A2,A3 . . . . . . be independent, every finite subset of the events must be

independent.

So pairwise independence is necessary but not sufficient.

Examples:

• One ticket is chosen at random from a box containing 4 lottery tickets with numbers

112, 121, 211, 222.

– The event Ai is that a 1 occurs in the ith place of the chosen number.

– P(Ai) = 12 i = 1, 2, 3 P(A1 ∩A2) = P({112}) = 1

4 Similarly for A1 ∩A3 and A2 ∩A3. These 3

events are therefore pairwise independent.

– Are they independent? No, since P(A1 ∩A2 ∩A3) 6= P(A1)P(A2)P(A3)

• Toss two dice, white and black. The sample space consists of all ordered pairs

(i, j) i, j = 1, 2 . . . 6. Define the following events :

– A1 : first die = {1, 2 or 3}, P(A1) = 12

– A2 : first die = {3, 4 or 5}, P(A2) = 12

– A3 : the sum of the faces equals 9, P(A3) = 19

In this case, P(A1 ∩A2 ∩A3) = P(3, 6) = 136 = ( 1

2)(12)(

19) = P(A1)P(A2)P(A3) but

P(A1 ∩A3) = P(3, 6) = 136 6= P(A1)P(A3) = 1

18 , so the events are not independent, nor

pairwise independent.



&

$

%

Conditional probability

• Probabilities reflect our beliefs about the likelihood of uncertain events. Conditional

probabilities reflect updated beliefs in the light of new evidence.

• All probabilities are conditional in that they reflect background knowledge.

Definition: If A and B are events with P(B) > 0, then the conditional probability of A, given B is defined as:

P(A|B) =P(A∩B)P(B)

A is the event whose uncertainty we wish to update and B is the evidence we observe or the event we want to

take as given. P(A) is called the prior probability of A and P(A|B) is the posterior or revised probability of A

To see why this makes sense:

• once we know B has occurred, we get rid of all elements of Bc since these contradict the

evidence

• we now redistribute their mass among the remaining outcomes in a way that preserves their

relative masses in B.



&

$

%

Conditioning on multiple events

For any 3 events, A1, A2 and A3 with positive probabilities,

P(A1,A2,A3) = P(A1)P(A2|A1)P(A3|A1,A2)

where commas denote intersections.

We can generalize this to as many events as we like. For A1,A2, . . .An:

P(A1,A2, . . .An) = P(A1)P(A2|A1)P(A3|A1,A2) . . .P(An|A1, . . . ,An−1)

Notice there are multiple forms of these expressions depending and which is most convenient to

use depends on the problem at hand.



&

$

%

Bayes’ Rule

Notice from the definition of conditional probability that the following expressions are equivalent:

P(A∩B) = P(A|B)P(B) = P(B|A)P(A)

We can therefore write

P(A|B) =P(B|A)P(A)

P(B)

This expression for conditional probability of event A, given B is known as Bayes’ Rule.



&

$

%

The law of total probability

Let A1,A2, . . .Ak be a partition of the sample space S, with P(Ai) > 0 for all i. Then

P(B) =

k∑i=1

P(Ai)P(B|Ai)

This is the law of total probability. Using conditional probabilities sometimes makes it easy to

solve a complicated problem.

Example: You play a game in which your score takes integer values between 1 and 50 with equal

probability. If your score the first time you play is equal to X, and you play until you score

Y ≥ X, what is the probability that Y = 50?

Solution: Let Ai be the event X = xi and B is getting a 50 to end the game. P(X = xi) = 150 . The

probability of getting xi in the first round and 50 to end the game is given by the product,

P(B|Ai)P(Ai). The required probability is the sum of these products over all possible values of i:

P(Y = 50) =

50∑x=1

1

51− x.1

50=

1

50(1+

1

2+

1

3+ · · ·+ 1

50) = .09



&

$

%

Bayes Rule ...examples

• C1, C2 and C3 are plants producing 10, 50 and 40 per cent of a company’s output. The

percentage of defective pieces produced by each of these is 1, 3 and 4 respectively. Given

that a randomly selected piece is defective, what is the probability that it is from the first

plant?

P(C1|C) =P(C|C1))(P(C1)

P(C)=

(.01)(.1)

(.01)(.1)+ (.03)(.5)+ (.04)(.4)=

1

32= .03

How do the prior and posterior probabilities of the event C1 compare? What does this tell

you about the difference between the priors and posteriors for the other events?

• Suppose that there is a new blood test to detect a virus. Only 1 in every thousand people

in the population has the virus. The test is 98 per cent effective in detecting a disease in

people who have it and gives a false positive for one per cent of disease free persons tested.

What is the probability that the person actually has the disease given a positive test result:

P(Disease|Positive) =P(Positive|Disease)P(Disease)

P(Positive)=

(.98)(.001)

(.98)(.001)+ (.01)(.999)= .089

So in spite of the test being very effective in catching the disease, we have a large number of

false positives. Why?



&

$

%

Bayes Rule ... priors, posteriors and politics

To understand the relationship between prior and posterior probabilities a little better, consider

the following example:

• A politician, on entering parliament, has a fairly good reputation. A citizen attaches a prior

probability of 34 to his being honest.

• At the end of his tenure, there are many potholes on roads in the politician’s constituency.

While these do not leave the citizen with a favorable impression of the incumbent, it is

possible that the unusually heavy rainfall over these years was responsible.

• Elections are coming up. How does the citizen update his prior on the moral standing of

the politician? Let us compute the posterior probability of the politician’s being honest,

given the event that the roads are in bad condition:

– Suppose that the probability of bad roads is 13 if the politician is honest and is 2

3 if

he/she is dishonest.

– The posterior probability of the politician being honest is now given by

P(honest|bad roads) =P(bad roads|honest)P(honest)

P(bad roads)=

( 13)(

34)

( 13)(

34)+ ( 2

3)(14)

=3

5

• What would the posterior be if the prior is equal to 1? What if it the prior is zero? What if

the probability of bad roads was equal to 12 for both types of politicians? When are

differences between priors and posteriors going to be large?



&

$

%

Conditioning matters: The Sally Clark case

• Sally Clark was a British solicitor who became the victim of a “one of the great miscarriages

of justice in modern British legal history”

• Her first son died within a few weeks of his birth in 1996 and her second one died in

similarly in 1998 after which she was arrested and tried for their murder.

• A well-known paediatrician Professor Sir Roy Meadow, who testified that the chance of two

children from an affluent family suffering sudden infant death syndrome was 1 in 73 million,

which was arrived by squaring 1 in 8500 for likelihood of a cot death in similar circumstance.

• Clark was convicted in November 1999. In 2001 the Royal Statistical Society issued a public

statement expressing its concern at the “misuse of statistics in the courts” and arguing that

there was “no statistical basis” for Meadow’s claim

• In January 2003, she was released from prison having served more than three years of her

sentence after it emerged that the prosecutor’s pathologist had failed to disclose

microbiological reports that suggested one of her sons had died of natural causes.

• Mistake: assumption of independence of outcomes, and confusing P(I|E) 6= P(E|I)(I=innocent, E=evidence). If the prosecution had looked at P(I|E), then the very large

prior on innocence P(I) would have played a role.



&

$

%

.

Conditional probability spaces

When we condition on an event B, then B becomes our sample space and conditional

probabilities replace our prior probabilities. In particular:

• P(S|B) = 1,P(∅|B) = 0

• If P(⋃)∞j=1Aj|B) =

∞∑j=1P(Aj|B) if the Aj are disjoint events

• P(AUE|B) = P(A|B)+P(E|B)−P(A∪B|E)

We can condition on as many events as we like when applying Bayes’ rule and the law of total

probability. If P(A∩B) and P(E∩B) are both greater than zero,

• Bayes’ Rule: P(A|E,B) =P(E|A,B)P(A|B)

P(E|B)

• LOTP: P(E|B) =k∑i=1P(E|Ai,B)P(Ai|B)

Conditional independence does not imply independence or vice-versa


Documents

Course 003: Basic Econometrics